Hypertext Transfer Protocol (HTTP)

Introduction#

The Hypertext Transfer Protocol (HTTP) is a stateless, application-layer protocol for distributed and hypermedia information systems. It's the foundation of data communication for the World Wide Web and is considered the de facto standard for client-server resource sharing. Web servers and client applications (browsers) must adhere to the message formats and transmission methods provided in the HTTP specifications. For instance, when we type a URL into a browser, the web server receives an HTTP request directing it to fetch and deliver the specified web page.

HTTP is based on the concepts of request and response, where one program (the client) requests another program (the server), and the server returns a response. The client usually makes requests in the form of HTTP commands, and the server responds with data in the form of documents. These commands are structured by different API architecture styles that we'll discuss in the coming chapters. Because it’s a driving force for APIs, HTTP is an essential protocol to understand.

As long as the request conforms to HTTP standards, the server will respond with the requested resources, regardless of the type of client.

This lesson discusses the essentials of HTTP. First, we explain its request and response formats. Next, we look into the HTTP methods, response status codes, and different HTTP headers. Toward the end, we discuss the significance of HTTP in APIs. Finally, we explain the secure version of HTTP, which is HTTPS.

HTTP message flow#

As discussed in the previous section, HTTP communication is initiated by the clients sending a request to the server, and the server responds. This section discusses the request and response message structure. The server parses each request to interpret the request and generate the response. In the same way, the client parses the received response from the server and collects the desired application data from the response message.

HTTP request structure#

An HTTP request message is composed of the following four components:

• Method: HTTP provides built-in methods that determine what kind of action a client wants to request/perform.

• The request-target and version: It can be a URL or URI, which is a unique address or identifier that determines a resource's location uniquely over the Internet. For example, the URL https://www.educative.io/courses/grokking-modern-system-design-interview-for-engineers-managers uniquely identifies an Educative course titled "Grokking Modern System Design Interview for Engineers & Managers." It also shows which version of HTTP is being used. The version part contains the HTTP version used for the request, for example, HTTP/1.1 or HTTP/2.0.

• Headers: HTTP headers allow clients to pass additional information between the communicating entities. These headers come in as a series of key-value pairs containing important information about each request. The request headers mainly contain the following information:

  • Server information from where data is requested.

  • Information about the browser being used by the user.

  • The kind of data formats a client can accept.

• Body: The body part of the HTTP request is used to communicate any application data a client wants to communicate to the server. The format of data should be mentioned in the header so that the server clearly understands the data.

A sample HTTP request is shown below, representing all four parts of an HTTP request.

HTTP response structure#

The HTTP response has the same format as the request with the following modifications:

• The method in the request is replaced with the HTTP version in response.

• The URL in the request is replaced with a status code in response.

• The HTTP version in the request is replaced with the phrase in response.

A sample HTTP response is shown below, including all three parts.

Note: Headers allow the exchange of extended control information between the client and server. There are a large number of headers available in the HTTP specification and they're extensively used.

HTTP methods#

The HTTP protocol is packed with several methods that are used by clients to request an action from the server. These methods are discussed below, along with respective curl commands to test them in the terminal on a publicly available domain, such as https://www.example.com.

Note: We should be aware of the fact that the names of these methods are case sensitive, so we should use upper case—for example, HEAD instead of head.

• The GET method: This method is used to read data from a web source, for example, reading a page from the server. We can understand how a GET request works by using the following command:

• The POST method: This method sends user-generated data to the server in order to create or update a data source. We can understand how the POST method works through the following command. Here, we pass a username, userA, and a password, SamplePassword, to the https://example.com/login page. However, in this case, it might give a 404 error (page not found) due to the unavailability of the login page.

• The PUT method: This method creates a new source or updates the existing one. It completely replaces whatever is present on the target URL. We can understand how the PUT method works by using the following command. Here, we’re updating the username to educative and email to edu@gamil.com.

• The DELETE method: This method deletes a resource from the server, for example, if we want to delete a user having a specific ID or name. In the following example, we delete a user having a username equal to user1:

• The TRACE method: This method echos the content of an incoming HTTP request. This method is used for debugging purposes. We can understand how the TRACE method works by using the following command:

• The CONNECT method: The connect method makes a two-way connection with a web server. It can be used to access a website using SSL or to open a tunnel. For instance, when a client wants to tunnel a TCP connection with a specific server (resource), they can ask the HTTP proxy server via the CONNECT method. The HTTP proxy server then establishes the connection for the client. We can try it with the following command:

• The OPTIONS method: This method queries options for a page supported by a web server. For example, we use the OPTIONS method if we want to see what HTTP methods a domain can support. Let's understand the working of the OPTIONS method in the following example:

• The HEAD method: This method asks for information about the document. It’s faster than the other methods because significantly less amount of information is being transferred. We can try the HEAD method in the terminal below using the following command:

We can try all the commands given above in the following terminal to see the working of each HTTP method.

We’ll see the following response to the HEAD method. The first line contains the HTTP version, the status code, and the phrase. The rest are some headers in the response.

The following table summarizes all the HTTP methods.

Note: HTTP also supports custom methods, which extend its functionality. For instance, WebDav provides a set of extensions to the HTTP protocol via custom methods such as PRPFIND, PRPPATCH, MOVE, LOCK, UNLOCK, and so on. These custom methods are used for distributed authoring and versioning of documents.

HTTP headers#

As discussed earlier, the request to a server may contain additional lines specifying extra information, which are called request headers. For example, Authorization is a request header that represents a list of a client's credentials. Similarly, a response can also contain headers from the server-side, for example, Server is a header that shows information about the server in response. The nature of the headers in the request and response may vary, but in some cases, these may be applicable to both request and response. For example, the header Content-Type can be a part of both request and response. Following are some HTTP headers that can be used in requests, responses, or both.

The significance of HTTP in APIs#

The communication ecosystem was already present in the form of HTTP, and APIs used that ecosystem to their advantage instead of inventing something new. The adaptation of HTTP by APIs brings the following advantages:

• The HTTP protocol is well-known to many developers, so it’s simple to use, upgrade, and scale.

• It allows cross-platform and cross-language communication to support interoperability.

• It provides a uniform interface for communication between entities in the form of request and response messages.

• It’s a fast protocol because it requires minimal processing.

• Due to its adaptability, APIs based on it can achieve a wide range of goals. For example, making custom requests based on the use case and utilizing different built-in methods provided by HTTP.

• Repeat requests can be handled more quickly because responses can be cached.

• HTTP supports various types of encoding techniques that are helpful in various types of communication with the server via APIs. The encoding techniques are carefully opted to make an API efficient.

Summary#

This lesson introduced a widely used client-server communication protocol known as HTTP. We described how an HTTP request and response are structured as well as learned about the commonly used HTTP methods. We also expanded on the different HTTP requests and response headers. Finally, we discussed the importance of HTTP in API design.